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A  PATH  MODEL  OF  U.S.  AIR  FORCE  PILOT  TRAINING  AND  ITS  ANTECEDENTS 


Malcolm  James  Ree 
Thomas  R.  Carretta 
Mark  S.  Teachout 

Summary 

A  causal  model  was  developed  that  investigated  the  role  of  general  cognitive  ability  (g)  and  prior 
job  knowledge  in  subsequent  job  knowledge  acquisition  and  work  sample  performance  during 
sequential  training.  Participants  were  3,428  U.  S.  Air  Force  officers  enrolled  in  a  53  week  pilot  training 
program.  The  measures  of  ability  and  prior  job  knowledge  came  from  the  Air  Force  Officer  Qualifying 
Test.  The  measures  of  job  knowledge  acquired  during  training  were  derived  from  pilot  training 
classroom  grades.  Work  sample  job  performance  measures  came  from  check  flight  ratings.  The  causal 
model  showed  that  ability  directly  influenced  the  acquisition  of  job  knowledge  both  prior  to  and  during 
training.  General  cognitive  ability  influenced  work  sample  performance  indirectly  through  job 
knowledge,  but  did  not  show  any  direct  influence.  This  may  have  occurred  because  all  participants  had 
a  low  level  of  job  (flying)  experience.  Prior  job  knowledge  had  almost  no  influence  on  subsequent  job 
knowledge,  but  directly  influenced  the  early  work  sample.  Early  training  job  knowledge  influenced 
subsequent  job  knowledge  and  work  sample  performance.  Finally,  early  work  sample  performance 
strongly  influenced  subsequent  work  sample  performance.  Implications  for  the  use  of  measures  of  g 
and  prior  job  knowledge  as  personnel  selection  variables  are  discussed. 

Introduction 

Several  studies  have  shown  the  effectiveness  of  general  cognitive  ability  (g)  as  a  predictor  of 
numerous  occupational  criteria  (e.g.,  McHenry,  Hough,  Toquam,  Hanson,  &  Ashworth,  1990; 
McNemar,  1964;  Ree  &  Earles,  1992).  McNemar  (1964),  and  Ree  and  Earles  (1991)  demonstrated 
that  g  accounted  for  almost  all  of  the  validity  of  multiple  aptitude  batteries  for  predicting  training 
success,  while  McHenry  et  al.  (1990)  and  Ree,  Earles,  and  Teachout  (1994)  demonstrated  that  g  was 
the  best  predictor  of  core  technical  job  performance.  Hunter  (1986)  provided  a  major  summary  article 
demonstrating  that  “...general  cognitive  ability  has  high  validity  predicting  performance  ratings  and 
training  success  in  all  jobs.”  (p.  359) 

Further,  Olea  and  Ree  (1994)  extended  these  findings  in  a  large  scale  study  of  about  5,400  U.  S. 
Air  Force  (USAF)  aviation  trainees  including  traditional  training  criteria  and  hands-on  work  samples  of 
flying  performance.  They  found  that  job  knowledge  was  incremental  (.08)  to  the  validity  of  g  (r  =  .314) 
for  pilots.  Humphreys  (1986)  observed  similar  results  during  World  War  II. 

Finally,  Dye,  Reck,  and  McDaniel  (1993)  demonstrated  the  validity  of  job  knowledge  tests  for 
many  jobs.  They  defined  job  knowledge  as  "...the  cumulation  of  facts,  principles,  concepts,  and  other 
pieces  of  information  that  are  considered  important  in  the  performance  of  one's  job."  (p.  153).  In  their 
meta-analysis  of  502  validity  coefficients  based  on  363,528  individuals,  they  found  a  mean  validity  of 
.47  for  predicting  training  performance  and  .45  for  predicting  job  performance. 
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In  addition  to  the  validity  of  g  and  job  knowledge,  their  causal  roles  in  job  performance  have  been 
demonstrated.  Hunter  (1983)  reported  a  path  analysis  of  meta-analytically  cumulated  correlations 
relating  g,  job  knowledge,  and  job  performance.  In  the  14  studies  with  3,264  participants  he  found  that 
the  major  causal  influence  of  cognitive  ability  was  on  the  acquisition  of  job  knowledge.  Job 
knowledge,  in  turn,  had  a  major  impact  on  work  sample  performance  and  supervisory  ratings.  Work 
sample  performance  also  directly  influenced  supervisory  ratings.  No  direct  effect  of  ability  on 
supervisory  job  performance  ratings  was  reported.  In  Hunter’s  model,  work  sample  performance  and 
job  knowledge  mediated  (James  &  Brett,  1984)  the  relationship  between  ability  and  supervisory 
ratings. 

Schmidt,  Hunter,  and  Outerbridge  (1986)  extended  Hunter  (1983)  to  include  experience  on  the 
job.  They  found  that  experience  influenced  measures  of  job  knowledge  and  work  sample  measures  of 
job  performance.  These  latter  two  measures  directly  influenced  supervisory  ratings  of  job  performance. 

Borman,  White,  Pulakos,  and  Oppler  (1991)  confirmed  Hunter’s  (1983)  model  in  an  additional 
sample  of  job  incumbents,  but  went  on  to  make  it  more  parsimonious  showing  sequential  effects  from 
ability  to  job  knowledge  to  task  proficiency  to  supervisory  ratings.  They  found  that  the  paths  from 
ability  to  task  proficiency  and  from  job  knowledge  to  supervisory  ratings  were  unnecessary.  Borman  et 
al.  (1991)  attributed  this  to  the  uniformity  of  job  experience  of  the  participants.  More  recently,  Borman, 
White,  and  Dorsey  (1995)  confirmed  the  Borman  et  al.  (1991)  parsimonious  model  on  peer  and 
supervisory  samples. 

Borman,  Hanson,  Oppler,  Pulakos,  and  White  (1993)  extended  the  model  to  supervisory  job 
performance  again  showing  that  ability  influenced  job  knowledge.  They  added  job  experience  as  a 
variable  and  significant  paths  between  experience  and  proficiency,  job  knowledge,  and  supervisory 
ratings  were  found.  Additionally,  a  path  between  ability  and  experience  was  found. 

The  current  study  investigated  the  causal  role  of  g_and  prior  job  knowledge  in  a  complex  sequential 
training  environment.  Others  have  explored  the  role  of  g  and  job  knowledge  in  job  performance.  We 
have  extended  this  to  training  performance  and  make  the  distinction  between  prior  job  knowledge  and 
job  knowledge  acquired  during  training.  Specifically,  we  investigated  a  model  of  the  causal  role  of  g 
and  prior  job  knowledge  in  acquiring  subsequent  job  knowledge  during  training  and  the  influence  of  g 
and  job  knowledge  on  work  sample  performance  during  training. 

The  hypotheses  in  the  current  study  are  based  on  previous  findings  (Borman  et  al.,  1991,  1993, 
1995;  Hunter,  1983;  Schmidt  et  al.,  1986).  The  current  study  is  like  Borman  et  al.  (1991,  1995) 
because  the  participants  have  similar  levels  of  experience.  In  addition,  Hunter  (1983)  and  Schmidt,  et 
al.  (1986)  suggested  that  the  direct  impact  of  g  on  performance  in  military  samples  would  be  small. 
They  suggested  this  because  of  the  highly  structured  nature  of  military  job  training.  Based  on  all  these 
previous  findings,  we  hypothesized  that  g  would  have  either  weak  or  no  direct  influence  on  the  work 
samples  and  that  g  would  exert  its  influence  indirectly  through  job  knowledge.  Further,  as  job 
knowledge  is  acquired  as  a  consequence  of  g,  prior  job  knowledge  would  be  a  direct  function  of  g. 
Additional  job  knowledge  should  be  a  direct  or  indirect  consequence  of  g.  In  the  circumstance  where 
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additional  job  knowledge  relied  heavily  on  preceding  job  knowledge,  the  effects  of  g  were 
hypothesized  to  be  indirect.  Prior  research  suggested  that  there  should  be  a  causal  link  between  job 
knowledge  and  work  sample  performance.  Also,  early  work  sample  performance  was  expected  to 
influence  subsequent  work  sample  performance  because  there  is  a  sequential  requirement  to  master 
one  to  advance  to  the  other.  Figure  A1  shows  the  hypothesized  model. 

Method 


Participants 

The  participants  were  3,428  USAF  officers  that  had  attended  and  completed  a  53  week  pilot 
training  course  between  the  years  1981  and  1993.  The  attrition  rate  for  this  course  varies  little  from  the 
long  term  average  of  22  %.  Although  the  exact  number  starting  the  training  program  is  not  maintained 
in  official  records,  we  estimate  that  the  size  of  the  sample  that  began  training  aggregated  across  the 
years  was  about  4,400.  The  sample  used  for  analyses  was  predominantly  male  (98.3%)  and  White 
(96.8%).  Participants  were  between  the  ages  of  about  22  and  27  years  and  all  had  completed  at  least  a 
baccalaureate  degree  from  an  accredited  college  or  university.  All  participants  had  been  selected  for 
officer  commissioning  and  for  pilot  training,  in  part,  on  the  basis  of  their  scores  on  the  Air  Force 
Officer  Qualifying  Test  (AFOQT).  The  Air  Force  uses  a  selection  board  technique  that  rates  applicants 
for  admission  into  flying  training  on  the  basis  of  academic  achievement  with  a  preference  toward 
science  majors.  Prior  flying  experience  in  some  cases,  medical  fitness,  and  personal  recommendations 
are  also  considered.  These  are  not  retained  in  archival  files  and  were  not  available  for  inclusion  in  this 
study. 

Measures 


g  and  Prior  Job  Knowledge.  The  measures  of  g  and  prior  job  knowledge  were  extracted  from  the 
AFOQT,  the  paper-and-pencil  test  used  to  select  students  for  officer  precommissioning  programs  (i.e., 
Officer  Training  School,  Reserve  Officer  Training  Corps)  and  to  qualify  commissioned  officers  for 
pilot  and  navigator  training  programs.  Like  other  military  tests,  the  AFOQT  is  based  on  a  detailed 
taxonomy  of  test  and  item  specifications.  This  taxonomy  defines  the  content  (Berger,  Gupta,  Berger,  & 
Skinner,  1990;  Gupta,  Berger,  Berger,  &  Skinner,  1989)  and  psychometric  properties  (Skinner  &  Ree, 
1987)  of  each  test. 

The  AFOQT  consists  of  16  tests  that  measure  g,  flying  job  knowledge,  and  four  lower-order 
cognitive  factors:  verbal,  quantitative,  spatial,  and  perceptual  speed  (Carretta  &  Ree,  1996).  For 
purposes  of  this  study,  verbal  and  quantitative  tests,  the  most  universally  accepted  measures  of  general 
cognitive  ability,  were  used  to  estimate  g.  Prior  job  knowledge  (JKP)  was  assessed  through  measures  of 
instrument  comprehension  and  aviation  information.  Descriptions  of  the  tests  grouped  by  content  are 
provided  below. 

Verbal  tests.  Verbal  Analogies  (VA)  measures  the  ability  to  reason  and  recognize  relationships 
between  words.  Reading  Comprehension  (RC)  assesses  the  ability  to  read  and  understand  paragraphs. 
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Word  Knowledge  (WK)  provides  a  measure  of  ability  to  understand  words  through  the  use  of 
synonyms. 

Quantitative  tests.  Arithmetic  Reasoning  (AR)  measures  the  ability  to  understand  arithmetic 
relationships  expressed  as  word  problems.  Data  Interpretation  (DI)  assesses  the  ability  to  extract 
information  from  data  presented  in  tables  and  charts.  Math  Knowledge  (MK)  requires  the  ability  to  use 
mathematical  terms,  formulas,  and  relationships  to  solve  problems.  Scale  Reading  (SR)  measures  the 
ability  to  read  and  extract  information  from  scales  and  dials. 

Prior  job  knowledge  tests.  These  are  the  only  tests  in  the  AFOQT  that  measure  specific  job 
knowledge  (Dye  et  al.,  1993;  Olea  &  Ree,  1994).  Instrument  Comprehension  (IC)  assesses  the  ability 
to  determine  the  attitude  (position  and  orientation  in  three  dimensional  space)  of  an  aircraft  in  flight 
based  on  information  presented  in  illustrations  of  flight  instruments.  Aviation  Information  (AI) 
measures  knowledge  of  general  aviation  principles,  concepts,  and  terminology. 

Pilot  Academic  and  Flying  Grades 

Pilot  academic  grades.  Academic  indicators  represented  student  pilots’  performance  on  written 
tests  of  flying  theory,  procedures,  and  aircraft-unique  systems  (i.e.,  hydraulics,  instruments,  electronics, 
etc.)  learned  during  training.  On  each  academic  test,  students  received  a  percent  correct  score.  There 
were  1 1  end-of-course  tests  (Al  through  All).  They  were  divided  into  three  groups  that  represented 
early  (Al  to  A4),  middle  (A5  to  A8),  and  late  training  (A9  to  Al  1).  Early  and  middle  classroom 
training  were  relevant  to  flying  the  subsonic  primary  aircraft  (T-37).  Early  classroom  training  included 
courses  in  T-37  systems,  aerospace  physiology/human  factors,  flying  fundamentals,  and  T-37 
aerodynamics.  Middle  classroom  training  provided  courses  relevant  to  flight  in  general  and  to  flying  the 
primary  aircraft  and  included  T-37  instruments  I  and  II,  T-37  navigation,  and  T-37  mission  planning. 
Late  classroom  training  was  relevant  to  the  supersonic  advanced  aircraft  (T-38)  including  T-38 
systems  operations,  applied  aerodynamics,  and  flight  planning. 

Flving  work  samples.  There  are  two  general  categories  of  flights  during  training  that  accumulate 
about  190  flying  hours.  On  ordinary  daily  flights  the  student  pilot  learns  and  practices  under  the 
watchful  eye  of  an  instructor  pilot.  After  the  prescribed  daily  flights,  work  sample  tests  called  check 
flights  are  rated  by  check  flight  pilots.  To  eliminate  potential  bias  due  to  familiarity,  check  flight  pilots 
do  not  rate  students  with  whom  they  have  flown  on  daily  flights. 

During  training,  student  pilots  completed  three  check  flights  in  the  primary  aircraft  (CF1  to  CF3) 
and  three  in  the  advanced  aircraft  (CF4  to  CF6).  In  the  primary  aircraft,  students  must  (1)  demonstrate 
the  ability  to  fly  to  a  geographical  area,  perform  maneuvers  and  return  to  perform  successful  landings, 
(2)  conduct  airborne  activities  within  precise  altitude  and  geographical  limits,  and  (3)  use  instruments 
with  an  emphasis  on  landing  approaches. 

The  advanced  training  aircraft  is  much  faster  than  the  primary  training  aircraft  and  all  activities 
must  be  accomplished  more  rapidly.  This  makes  even  familiar  maneuvers  more  difficult  in  the 
advanced  aircraft.  The  check  flights  for  round  trips  to  geographical  areas  and  instruments  are  similar  to 
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the  check  flights  in  the  primary  aircraft.  The  difficult  formation  check  flight  is  added  where  the  wings 
of  multiple  aircraft  are  as  close  as  three  feet  at  speeds  of  400  knots.  See  Duke  and  Ree  (in  press)  for  a 
complete  description. 

Each  check  flight  grade  score  was  a  weighted  average  of  ratings  of  several  flying  procedures  and 
maneuvers.  These  procedures,  maneuvers,  and  weights  are  prescribed  by  the  Air  Force  in  training 
regulations.  The  student  pilot  receives  points  for  each  procedure  accomplished.  Example  procedures 
are:  retract  landing  gear  at  specified  speed,  make  proper  radio  calls  during.flight,  or  perform  a  loop 
within  specified  parameters  (e.g.,  engine  power  settings  and  maneuver  entry  altitude).  As  with 
academic  grades,  check  flight  grades  were  percentage  scores. 

The  sequential  training  environment  was  structured  as  follows.  Theory  and  general  background 
were  taught  first  in  the  classroom  followed  by  application  in  the  aircraft.  Classroom  training  for  the 
primary  aircraft  began  prior  to  check  flight  work  samples.  The  final  check  flight  work  sample  in  the 
primaiy  aircraft  was  completed  after  the  last  classroom  instruction  in  middle  training  (A5  to  A8).  After 
the  final  check  flight  in  the  primary  aircraft,  classroom  instruction  relevant  to  the  advanced  aircraft 
began  and  was  followed,  shortly  thereafter,  by  the  check  flight  work  samples.  The  last  check  flight 
work  sample  in  the  advanced  aircraft  occurred  after  all  the  classroom  training  had  been  completed. 

Procedures 


Measures  of  g  and  job  knowledge  acquired  prior  to  training  were  included  as  were  sequentially- 
ordered  blocks  of  classroom  training  and  hands-on  work  sample  performance  measures.  Because  the 
participants  had  been  selected,  at  least  in  part,  on  the  basis  of  the  scores  on  the  test  battery  that 
yielded  the  estimates  of  g  and  prior  job  knowledge,  they  constituted  a  censored,  range  restricted 
sample.  Such  samples  provide  relatively  poor  statistical  estimates  of  the  correlations  among 
variables  (Thorndike,  1949).  We  used  the  multivariate  method  (Lawley,  1943;  Ree,  Carretta,  Earles, 
&  Albert,  1994)  to  correct  the  correlation  matrix  for  the  consequences  of  prior  selection.  The 
corrected  matrix  was  used  in  the  structural  equation  analyses. 

The  Bentler-Weeks  (Bentler  &  Weeks,  1980)  structural  model  was  estimated  using  maximum 
likelihood  procedures  as  implemented  in  version  4.02  of  the  EQS  program.  This  program 
corrects  for  unreliability  via  estimation  in  the  same  fashion  as  LISREL  and  most  other  structural 
modeling  programs.  The  estimated  reliabilities  can  either  be  provided  as  starting  values  or  they 
can  be  estimated  directly  from  the  data.  In  these  analyses,  they  were  estimated  directly  from  the 
data. 

We  followed  the  rules  offered  by  Marsh  (1994)  as  a  general  approach  to  evaluating  goodness-of- 
fit.  These  are:  (1)  determine  that  the  iterative  procedure  converged  to  a  proper  solution,  (2)  establish 
that  parameter  estimates  were  reasonable  in  accordance  with  the  a  priori  model  and  common  sense, 
and  (3)  evaluate  the  various  fit  indexes  in  relation  to  rules  of  thumb  and  values  from  competing 
models. 
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We  considered  several  goodness-of-fit  statistics  and  chose  the  Comparative  Fit  Index  (CFI; 
Bender,  1989)  that  has  been  shown  to  have  little  sample  size  dependence  and  a  small  sampling 
variance.  The  model  %2  and  root  mean  square  error  of  approximation  (RMSEA)  also  are  presented. 
Finally,  the  residuals  were  inspected  to  determine  their  magnitude  and  to  determine  if  any  variable  was 
being  poorly  predicted  by  the  model. 

We  first  fit  the  measurement  model  and  then  the  path  (causal)  models.  Because  the  scales  of  the 
variables  are  not  well  known  or  intrinsically  meaningful,  we  reported  the  path  coefficients  as 
standardized  regression  coefficients  (Cohen  &  Cohen,  1983).  These  coefficients  should  be  interpreted 
as  indicating  that  a  standard  deviation  change  in  an  independent  variable  leads  to  a  change  in  the 
dependent  variable  equal  to  the  magnitude  of  the  path  coefficient. 

The  causal  model  (Figure  A2)  based  on  Hunter  (1983)  and  Schmidt  et  al.  (1986)  with  all  the 
hypothesized  links  was  estimated.  It  was  compared  with  a  revised  model  (Figure  A3)  suggested  by  the 
findings  of  Borman  et  al.  (1991).  We  computed  the  squared  multiple  correlations  (R  )  for  dependent 
variables  for  dependent  variables. 

Results 

Tables  1  and  2  show  the  observed  and  unrestricted  means,  standard  deviations,  reliabilities,  and 
correlations  for  all  variables. 

The  measurement  model  is  shown  in  Table  3.  Loadings  of  the  tests  and  other  measurement 
variables  can  be  found  in  this  table.  The  aptitude  tests  loaded  on  lower-order  verbal  (V)  and 
quantitative  (Q)  factors  with  g  in  hierarchical  position.  The  specific  job  knowledge  tests  lead  to  a  prior 
job  knowledge  (JKP)  factor.  Job  knowledge  acquired  during  training  was  represented  by  factors 
derived  from  early  (JKTi),  middle  (JKT2),  and  late  (JK-n)  training  as  measured  by  job  knowledge  tests. 

Two  work  sample  performance  factors  (WSi  and  WS2)  were  derived  from  the  six  check  flights,  three 
in  each  aircraft  type.  Latent  factor  correlations  among  g,  prior  job  knowledge,  job  knowledge  acquired 
during  training,  and  work  samples  are  shown  in  Table  4. 

The  measurement  model  fit  the  data  well.  The  CFI  was  .969  which  is  very  close  to  the  maximum 
possible  value  of  1.  The  model  %2  was  1,075.267  (df  =271)  and  was  appreciably  smaller  than  the 
independence  %2  of  26,277.069  (df  =  325).  The  RMSEA  was  .034  which  Browne  and  Cudeck  (1993) 
place  in  the  "good"  category.  Additionally,  the  average  absolute  standardized  residual  was  only  .019, 
indicating  good  fit. 

The  hypothesized  path  model  based  on  Hunter  (1983)  and  Schmidt  et  al.  (1986)  was  computed. 

Its  estimated  path  coefficients  are  shown  in  Figure  A2.  The  fit  was  good.  The  CFI  was  .968,  the 
RMSEA  was  .034,  and  the  average  absolute  standardized  residual  was  .020.  However,  there  were 
several  non-significant  paths  (g  — >  WSi,  g  — »WS2,  g— »JKt2,  JKp— »JKti,  and  JKt3— >WS2).  Figure  A3 
presents  the  revised  model  and  its  path  coefficients.  Based  on  the  findings  of  Hunter  (1983)  and 
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Table  1.  Means  and  Standard  Deviations  for  AFOQT  Tests,  Academic  Grades  and  Check  Flight 
Grades 


Restricted  (Observed) 

Unrestricted 

Scorea 

Mean  SD 

Mean  SD 

Reliability15 

VA 

15.29 

3.36 

13.36 

4.23 

.80 

AR 

13.54 

4.11 

11.00 

4.40 

.81 

RC 

17.44 

4.73 

15.83 

5.93 

.88 

DI 

13.52 

3.91 

11.15 

3.93 

.71 

WK 

13.96 

5.17 

13.28 

5.83 

.88 

MK 

18.01 

4.63 

14.48 

6.04 

.88 

SR 

24.23 

5.54 

20.07 

6.73 

.84 

IC 

13.66 

4.23 

8.82 

4.76 

.84 

AI 

11.74 

4.26 

8.65 

4.08 

.77 

Al 

97.46 

3.11 

96.60 

3.19 

.23 

A2 

97.18 

3.32 

96.52 

3.38 

.20 

A3 

97.04 

3.37 

96.40 

3.41 

.18 

A4 

98.06 

3.28 

97.19 

3.36 

.22 

A5 

95.99 

4.80 

95.01 

4.89 

.24 

A6 

95.18 

5.32 

94.28 

5.37 

.20 

A7 

94.77 

5.36 

93.64 

5.43 

.20 

A8 

95.88 

4.55 

95.11 

4.60 

.19 

A9 

97.37 

3.32 

96.76 

3.38 

.12 

A10 

97.30 

3.63 

96.64 

3.70 

.21 

All 

96.83 

3.69 

96.04 

3.76 

.20 

CF1 

86.59 

7.56 

85.12 

7.60 

.23 

CF2 

90.65 

5.73 

89.59 

5.77 

.24 

CF3 

93.59 

4.87 

92.61 

4.93 

.26 

CF4 

91.21 

5.70 

90.07 

5.74 

.18 

CF5 

92.65 

4.67 

91.89 

4.69 

.15 

CF6 

93.82 

4.73 

92.89 

4.78 

.17 

Notes.  “The  first  9  variables  are  tests  from  the  AFOQT:  VA  =  Verbal  Analogies,  AR  =  Arithmetic 
Reasoning,  RC  =  Reading  Comprehension,  DI  =  Data  Interpretation,  WK  =  Word  Knowledge,  MK  =  Math 
Knowledge,  SR  =  Scale  Reading,  IC  =  Instrument  Comprehension,  AI  =  Aviation  Information.  A1  through 
All  are  the  1 1  job  knowledge  tests  and  CF1  through  CF6  are  the  6  check  flight  ratings. 
bReliabilities  for  the  AFOQT  tests  were  taken  from  Sinner  and  Ree  (1987),  the  sample  to  which  the  data 
were  corrected  for  range  restriction.  Reliabilities  for  the  job  knowledge  scores  and  check  flight  ratings  were 
the  values  estimated  by  EQS  in  the  measurement  model. 
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Table  2.  Correlation3  of  AFOQT  Tests,  Academic  Grades  and  Check  Flight  Grades 


vo 

tin 

U 

in 

U 

£ 

u 

co 

Uh 

u 

£ 

u 


u< 

u 


< 

© 

< 

ov 

< 


oo 

< 


r~~ 

< 


VO 

< 

»n 

< 

< 

eo 

< 


< 

U 


PC 

A 


2 

£ 


u 

PC 

oc 

< 

< 

> 


£ 

00 

VO 

o 

CM 

CM 

<n 

CO 

1-4 

00 

00 

VO 

c- 

CM 

CM 

Ov 

OV 

Tf 

c- 

r- 

Ov 

VO 

m 

o 

o 

o 

O 

o 

'  ' 

o 

o 

O 

o 

O 

o 

o 

o 

O 

4—1 

4— 

— i 

'O 

co 

VO 

o 

m 

vO 

os 

CN 

Tt 

in 

m 

rt 

Ov 

00 

00 

VO 

c- 

in 

Ov 

Os 

Ov 

o 

VO 

© 

o 

o 

o 

o 

o 

o 

O 

© 

o 

o 

o 

o 

O 

o 

© 

o 

o 

o 

o 

CM 

4-4 

4™ ' 

2 

4— 4 

4-4 

VO 

CM 

00 

— 

r- 

o 

o 

m 

VO 

VO 

m 

o 

r- 

o 

Ov 

00 

2 

CO 

OV 

4-4 

Ov 

Os 

o 

Ov 

00 

o 

o 

O 

o 

o 

1 

o 

o 

o 

© 

o 

o 

O 

o 

o 

CM 

1—4 

o 

vo 

Os 

vO 

o 

ff) 

CM 

oo 

CM 

r- 

-rt 

o 

vO 

CM 

o 

o 

o 

,  , 

CO 

m 

00 

o 

o 

o 

o 

o 

O 

o 

o 

o 

o 

4-4 

CM 

CM 

o 

CM 

CM 

CM 

m 

00 

co 

o 

CN 

r- 

os 

m 

Ov 

r- 

vO 

00 

Tf 

o 

CM 

, 

CO 

, 

in 

o 

Ov 

o 

O 

00 

o 

o 

O 

o 

o 

o 

o 

© 

o 

O 

o 

— 

4-4 

CM 

o 

CM 

CM 

CM 

— 

c* 

o 

00 

© 

in 

o 

o 

00 

r- 

in 

c** 

ov 

rf 

CO 

, 

o 

o 

CM 

CM 

o 

VO 

VO 

CM 

CM 

Ov 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

— 

4— 

o 

CM 

CM 

CM 

CM 

<N 

m 

CM 

VO 

oo 

00 

m 

VO 

in 

Tt 

r- 

<n 

CM 

r- 

oo 

o 

00 

o 

O 

o 

Tf 

CO 

m 

r- 

i 

o 

o 

o 

■ 

4-1 

CM 

*— 4 

CM 

CM 

o 

4-4 

4— 1 

o 

4— 1 

co 

Os 

m 

Tj- 

CM 

m 

in 

<n 

Os 

m 

^t 

VO 

00 

00 

O' 

r- 

—i 

O 

o- 

rt 

in 

tj- 

m 

Ov 

vO 

CM 

o 

O 

1-4 

1— 4 

— .4 

CM 

o 

4-4 

CM 

4—4 

4-4 

— 

o 

m 

CM 

VO 

vO 

r- 

Os 

VO 

o 

Ov 

CO 

CO 

CO 

CM 

O 

CO 

CO 

CO 

CM 

VO 

CM 

i—i 

o 

o 

CM 

CM 

CM 

CM 

CM 

o 

CM 

CM 

4-4 

4-t 

o 

4—4 

Tf 

CM 

o 

CM 

VO 

00 

CCi 

c- 

in 

m 

Tf 

ov 

O 

O 

Ov 

O 

CO 

CO 

Ov 

c~ 

Ov 

o 

o 

CM 

CM 

o 

CM 

4-4 

CM 

1 

—4 

O 

o 

O 

2 

OO 

CM 

Os 

o 

m 

m 

r- 

© 

rf 

00 

VO 

Ov 

00 

0 

CM 

VO 

c- 

CM 

CM 

CM 

CM 

o 

CM 

— i 

♦— i 

o 

CM 

2 

CM 

CM 

CM 

4-4 

oo 

Tfr 

Os 

CM 

t*-. 

CM 

in 

r- 

oo 

r- 

Ov 

O 

. 

0 

o 

»n 

o 

O 

m 

CM 

ov 

o 

1“l 

© 

© 

1—1 

r-4 

o 

o 

CM 

CM 

2 

CM 

CM 

CM 

CM 

CM 

4-1 

1 

1—1 

o 

4-4 

CM 

VO 

CM 

vO 

VO 

in 

cn 

O 

Os 

, 

CM 

O 

CO 

O' 

r- 

vO 

o 

VO 

CO 

00 

Ov 

^4 

in 

1-H 

O 

o 

o 

CM 

CM 

o 

CM 

CM 

CM 

CM 

CM 

CM 

4-4 

4—1 

o 

4-4 

1—1 

tT 

VO 

VO 

Tf 

r- 

1— 1 

m 

r- 

O 

VO 

CM 

CM 

VO 

CO 

O 

VO 

o 

CO 

CM 

vO 

o 

»— 1 

CM 

CM 

o 

CM 

CM 

CM 

CM 

4-4 

o 

r- 

m 

VO 

(N 

Os 

Os 

Os 

m 

o 

CO 

t 

Ov 

c* 

CM 

r- 

vO 

OO 

00 

VO 

C> 

VO 

Ov 

1 

' * 

■  1 

o 

o 

o 

CM 

CM 

CM 

CM 

4-4 

o 

o 

4-4 

o 

© 

O 

r» 

VO 

CM 

in 

in 

O 

Os 

o 

Os 

o 

00 

CM 

O 

O 

o 

Ov 

00 

o 

vO 

OV 

o 

oo 

VO 

w~* 

O 

,-4 

o 

CM 

CM 

CM 

CM 

CM 

4— 1 

t— 1 

CM 

o 

O 

© 

o 

4-4 

VO 

p- 

t- 

vO 

m 

oo 

Tf 

(N 

m 

O 

<n 

CM 

C' 

o 

C- 

VO 

CO 

CO 

00 

Ov 

in 

ov 

VO 

w~* 

,—l 

»— i 

O 

CM 

CM 

CM 

CM 

CM 

r-4 

CM 

CM 

— 

O 

— 

© 

© 

— 

r- 

CM 

,  , 

T__ 

r-» 

o 

00 

m 

o 

o 

VO 

OV 

o 

. 

m 

CM 

CO 

O 

CM 

O 

Ov 

Ov 

00 

in 

vO 

(N 

CM 

CM 

o 

© 

CM 

— H 

"■" 

*— i 

— i 

O 

O 

o 

o 

© 

c- 

O 

co 

in 

VO 

00 

vO 

0 

VO 

Os 

in 

Tt 

Ov 

CM 

o 

CM 

1—4 

CM 

CM 

CO 

CO 

CO 

CM 

CO 

CM 

co 

CM 

co 

CM 

m 

2 

>n 

4— 

4— 

co 

m 

CM 

Ov 

CN 

© 

Ov 

m 

cn 

00 

00 

. 

CM 

c- 

Ov 

o 

Os 

CM 

CO 

rf 

VO 

ov 

m 

o 

00 

co 

in 

co 

m 

CM 

o 

Tf 

cn 

CM 

CM 

CM 

CM 

y~l 

CM 

CM 

— 

1 

’“■* 

4— 

Ov 

r- 

00 

co 

cn 

o 

© 

Ov 

m 

00 

m 

CM 

r- 

vO 

CO 

m 

CM 

r- 

o 

00 

00 

1-H 

O 

CM 

c- 

OV 

co 

m 

co 

co 

o 

vO 

m 

CM 

CM 

CM 

CM 

CM 

— 4 

CM 

4-4 

o 

1—4 

4—4 

© 

o 

o 

CM 

ov 

Tt 

o 

o 

r- 

oo 

CM 

00 

O 

VO 

c- 

. 

o 

CM 

r- 

»n 

CM 

O 

Tt 

VD 

o 

VO 

VO 

vO 

o 

<n 

CM 

CM 

4-4 

O 

o 

© 

o 

o 

O 

c* 

co 

co 

o 

vO 

o 

(N 

m 

m 

Ov 

OV 

CC 

Ov 

c- 

CO 

Ov 

o 

4— 

CO 

4-4 

VO 

CO 

Ov 

vO 

VO 

© 

VO 

VO 

cn 

CM 

CM 

*— 4 

CM 

CM 

CM 

CM 

1 

4— 

o 

oo 

Tj- 

o 

in 

r-. 

»n 

rj- 

-rt 

m 

OV 

00 

o* 

c- 

00 

CM 

00 

CM 

t— 

in 

CO 

m 

^r 

o 

m 

t> 

in 

m 

cn 

CM 

CM 

CM 

1—4 

CM 

CM 

o 

O 

1—1 

o 

o 

oo 

o 

00 

r- 

vO 

T-1 

VO 

VO 

CM 

m 

>n 

CO 

Ov 

CO 

o 

CM 

C** 

CO 

o 

CO 

m 

4-4 

r- 

in 

^r 

o 

m 

vo 

rf 

r*- 

vO 

m 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

CM 

4-4 

o 

o 

00 

co 

co 

00 

m 

OO 

Tf 

o 

'rj* 

m 

Ov 

m 

© 

O' 

C- 

OO 

CM 

O 

Ov 

CO 

Os 

CM 

in 

VO 

o 

>n 

r- 

m 

VO 

in 

m 

CM 

CM 

CM 

CM 

— 

CM 

CM 

4-4 

O 

O 

— 

o 

o 

— 

o 

o 

CO 


<  as 
>  < 


u 

0$ 


HH  iTj 

Q  £ 


s 


(j 

00  HH 


< 


— •  cm  co 

<  <  < 


tJ-  m  vO 

<  <  < 


0^-(Mn-<tinvo 
r^ooav  —  —  b  ^  b  b  Ph  ^ 
<<<<<UUUUUU 


8 


Notes.  Correlations  above  the  diagonal  are  observed  data.  Those  below  the  diagonal  were  corrected  for  range  restriction  (Lawley,  1943). 
bThe  first  9  variables  are  tests  from  the  AFOQT:  VA  =  Verbal  Analogies,  AR  =  Arithmetic  Reasoning,  RC  =  Reading  Comprehension, 
DI  =  Data  Interpretation,  WK  =  Word  Knowledge,  MK  =  Math  Knowledge,  SR  =  Scale  Reading,  IC  =  Instrument  Comprehension, 

AI  =  Aviation  Information.  A1  through  All  are  the  1 1  job  knowledge  tests  and  CF1  through  CF6  are  the  6  check  flight  ratings. 


Table  3.  Factor  Loadings  of  the  Measurement  Model 


Factorb 


Score3  V  Q  g 


VA 

.46 

.68 

AR 

.51 

.83 

RC 

.60 

.67 

DI 

.03 

.61 

WK 

.68 

.54 

MK 

.13 

.77 

SR 

.05 

.76 

IC 

AI 

A1 

A2 

A3 

A4 

A5 

A6 

A7 

A8 

A9 

A10 

All 

CF1 

CF2 

CF3 

CF4 

CF5 

CF6 


JKP 


.86 

.65 


JKT1  JKT2  JKt3  WS,  WS2 


.51 

.46 

.44 

.50 

.52 

.48 

.47 

.46 

.52 

.48 

.47 

.51 

.52 

.55 

.44 

.41 

.43 


Notes.  “The  first  9  variables  are  tests  from  the  AFOQT:  VA  =  Verbal  Analogies, 
AR  =  Arithmetic  Reasoning,  RC  =  Reading  Comprehension,  DI  =  Data 
Interpretation,  WK  =  Word  Knowledge,  MK  =  Math  Knowledge,  SR  =  Scale 
Reading,  IC  =  Instrument  Comprehension,  AI  =  Aviation  Information. 

Al  through  AI  1  are  the  1 1  job  knowledge  tests  and  CF1  through  CF6  are  the 
6  check  flight  ratings. 

^he  factors  are  V  =  Verbal,  Q  =  Quantitative,  g  =  general  cognitive  ability, 

JKp  =  prior  job  knowledge,  JKti  =  job  knowledge  acquired  during  training 
(measure  1),  JKT2  =  job  knowledge  acquired  during  training  (measure  2), 

JKT3  =  job  knowledge  acquired  during  training  (measure  3),  WSj  =  flying  training 
work  sample  (measure  1),  and  WS2  =  flying  training  work  sample  (measure  2). 
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Table  4.  Correlations  Between  Factors'1  in  the  Causal  Model 


Factor 

g 

JKP 

jkt1 

JKT2 

JKt3 

WS, 

ws2 

£ 

1.00 

JKP 

.62 

1.00 

jkt1 

.63 

.45 

1.00 

JKt2 

.55 

.30 

.87 

1.00 

JKt3 

.61 

.33 

.86 

.95 

1.00 

ws, 

.33 

.29 

.44 

.56 

.54 

1.00 

ws2 

.37 

.35 

.44 

.54 

.56 

.92 

1.00 

Note.  “The  factors  are  V  =  Verbal,  Q  =  Quantitative,  g  =  general 
cognitive  ability,  JKP  =  prior  job  knowledge,  JKTi  =  job  knowledge 
acquired  during  training  (measure  1),  JKT2  =  job  knowledge  acquired 
during  training  (measure  2),  JKT3 = job  knowledge  acquired  during 
training  (measure  3),  WSi  =  flying  training  work  sample  (measure  1), 
and  WS2  =  flying  training  work  sample  (measure  2). 

Table  5.  Variance  Accounted  for  in  the  Dependent 
Variables 


R i 


Variable3 

Initial 

Model 

Revised 

Model 

JKP 

.38 

.38 

JKti 

.40 

.40 

JKt2 

.74 

.74 

JKt3 

.94 

.94 

WS, 

.32 

.31 

WS2 

.86 

.87 

Note.  “The  factors  are  JKP  =  prior  job  knowledge, 
JKTi  =  job  knowledge  acquired  during  training 
(measure  1),  JKn  =  job  knowledge  acquired 
during  training  (measure  2),  JKT3  =  job  knowledge 
acquired  during  training  (measure  3),  WSi  =  flying 
training  work  sample  (measure  1),  and 
WS2  =  flying  training  work  sample  (measure  2). 
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Schmidt  et  al.  (1986)  the  direct  paths  from  g  to  the  work  samples  were  removed.  There  was  no 
theoretical  basis  for  removing  the  other  non-significant  paths  and  they  were  retained.  All  of  the  fit 
indexes  stayed  the  same  as  in  the  initial  model,  indicating  no  loss  in  fit  due  to  removal  of  paths.  The 
proportion  of  variance  accounted  for  (R2)  for  each  dependent  variable  for  both  causal  models  is 
presented  in  Table  5. 


Discussion 

We  conducted  a  longitudinal  study  to  investigate  the  causal  roles  of  g  and  prior  job  knowledge  in  a 
complex,  sequential  training  environment.  While  the  initial  model  fit  the  data  quite  well,  some  paths 
were  non-significant.  The  revised  model  required  fewer  parameters  and  showed  no  decrease  in  fit. 
Additionally,  it  was  closer  to  a  model  proposed  by  Borman  et  al.  (1991,  1995).  The  reason  the  revised 
model  was  more  appropriate  for  these  data  is  probably  due  to  the  thoroughness  of  detailed  military 
training  programs  and  strict  adherence  to  standard  operating  procedures  (Hunter,  1983;  Schmidt  et  al., 
1986).  In  the  military,  performance  requirements  are  laid  out  in  great  detail  in  technical  manuals  and 
training  programs.  Hence,  in  military  samples,  ability  is  less  likely  to  have  a  direct  impact  on 
performance.  It  is  more  likely  that  ability  will  have  an  indirect  impact  on  performance  through  job 
knowledge  as  found  in  the  current  study. 

In  the  revised  model,  g  was  found  to  influence  work  sample  performance  only  indirectly  through 
prior  job  knowledge  and  job  knowledge  acquired  during  training.  This  was  consistent  with  and  extends 
past  research  that  has  shown  the  relationship  of  g  to  job  performance  (Borman  et  al.,  1991,  1993,  1995; 
Hunter,  1983;  Schmidt  et  al.,  1986). 

In  the  revised  model,  g  led  directly  to  the  acquisition  of  prior  job  knowledge  (JKp)  and  job 
knowledge  acquired  during  early  (JKTi)  and  late  training  (JKn).  Surprisingly,  the  direct  path  from  g  to 
middle  job  knowledge  (JKn)  was  near  zero.  The  causal  influence  of  g  on  JKt2  was  mediated  by  JKP 
and  JKti. 

A  direct  path  from  g  to  work  sample  performance  was  not  necessary.  The  influence  of  g  on  work 
samples  was  entirely  mediated  by  job  knowledge.  This  was  expected  because  g  leads  to  job  knowledge 
and  job  knowledge  leads  to  job  performance,  particularly  in  military  contexts  where  standard  operating 
procedures  are  detailed  in  technical  manuals  and  training  programs. 

Job  knowledge  acquired  in  early  training  influenced  subsequent  job  knowledge  acquisition  and 
work  sample  performance.  Early  training  job  knowledge  generally  dealt  with  the  basics,  while  later  job 
knowledge  added  information  about  specific  aircraft  systems  (hydraulics,  instruments,  unique  landing 
requirements,  etc.).  The  course  content  progression  was  from  general  to  specific.  The  paths  from  JKTi 
to  JKT2  and  from  JKT2  to  JKT3  were  about  equal  and  quite  strong.  Some  of  the  magnitude  of  these 
paths  may  have  come  from  the  fact  that  early  courses  are  foundations  for  subsequent  courses.  In  JKti, 
the  course  in  aerodynamics  is  a  prerequisite  for  the  applied  aerodynamics  course  in  JKt3.  Also  in  JKti, 
the  systems  course  is  prerequisite  to  the  instruments  I  and  II  courses  in  JKT2-  Similarly,  the  navigation 
course  and  mission  planning  course  in  JKt2  are  prerequisites  for  the  flight  planning  course  in  JKt3. 
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Finally,  flying  fundamentals  in  JKn  is  a  building  block  for  several  of  the  courses  in  both  JKt2  and 

JKT3. 


The  combined  direct  and  indirect  paths  from  job  knowledge  to  early  work  sample  performance 
(WSi)  were  about  equal  to  those  found  in  previous  research  when  experience  was  held  constant.  The 
direct  path  of  the  late  training  job  knowledge  (JKt3)  to  WS2  was  small.  The  indirect  paths  through  WSi 
were  much  larger. 

The  single  direct  path  from  WSi  to  WS2  was  very  strong  indicating  a  powerful  causal  role  for  skills 
and  experience  acquired  in  early  samples  of  job  performance.  Part  of  the  strength  of  this  path  may  have 
been  because  the  activities  learned  flying  the  primary  aircraft  were  required  in  flying  the  advanced 
aircraft.  Both  required  preflight  preparation,  take-offs,  aeronautic  maneuvers,  landings,  and  other 
elements. 

In  this  study  there  was  no  final  dependent  measure  equivalent  to  the  supervisory  ratings  reported 
by  Hunter  (1983),  Schmidt  et  al.  (1986),  and  Borman  et  al.  (1991,  1993, 1995).  However,  WSi  is 
comparable  to  the  variable  called  “work  sample”  in  Hunter  (1983)  and  in  Schmidt  et  al.  (1986),  “task 
proficiency”  in  Borman  et  al.  (1991),  “supervisory  proficiency”  in  Borman  et  al.  (1993),  and  “technical 
proficiency”  in  Borman  et  al.  (1995). 

We  have  demonstrated  the  causal  role  of  general  cognitive  ability  and  prior  job  knowledge  in  a 
complex  training  environment.  The  causal  role  of  g  is  through  the  acquisition  of  job  knowledge  that  in 
turn  has  an  impact  on  work  sample  performance.  The  causal  impact  of  prior  job  knowledge  was  very 
weak.  Its  influence  on  early  work  sample  performance  was  greater  than  its  influence  on  job  knowledge 
acquired  during  training.  Job  knowledge  acquired  during  training  had  a  strong  impact  on  job 
knowledge  acquired  later  in  training,  but  only  moderate  to  weak  influence  on  work  sample 
performance.  Not  surprisingly,  learning  to  fly  the  primary  aircraft  was  highly  influential  in  learning  to 
fly  the  advanced  aircraft. 

Selecting  applicants  with  high  scores  on  g  and  high  scores  on  prior  job  knowledge  should  lead  to 
better  performance  in  training  and  thereby  better  performance  on  the  job  as  shown  in  previous  research 
(Borman  et  al.,  1991,  1993;  Hunter,  1983;  Olea  &  Ree,  1994;  Schmidt  et  al.  1986,  1988).  However, 
the  causal  influence  of  prior  job  knowledge  was  very  small  compared  to  that  of  g.  This  suggests  that 
given  competition  for  resources,  testing  of  general  cognitive  ability  might  yield  greater  gains  and 
should  take  precedence  over  testing  for  prior  job  knowledge. 
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APPENDIX 

Three  Models  for  Sequential  Training 
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Figure  A2.  Initial  Causal  Model  and  Path  Coefficients  for  Sequential  Training 
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Figure  A3.  Revised  Causal  Model  and  Path  Coefficients  for  Sequential  Training 
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